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GPS Education Partners (GPSEd) is a community-based nonprofit organization and work-based 
learning intermediary offering a statewide manufacturing youth apprenticeship in Wisconsin. 
During their one to two years in the program, students meet high school graduation requirements, 
receive industry-specific and job-specific training, technical certifications, and Department of 
Workforce Development Youth Apprenticeship Certification. This report describes a research 
collaboration between GPSEd and ACT that examined the relationships between ACT® 
WorkKeys® performance and outcomes for GPSEd students. Analyses revealed that WorkKeys 
scores correlated with academic outcomes like reading ability level and math course-taking 
patterns. WorkKeys scores and WorkKeys® National Career Readiness Certificate® (NCRC®)' 
Levels were also related to Manufacturing Skill Standards Council (MSSC) Certified Production 
Technician (CPT) certificate attainment as well as attendance in GPSEd classes and 
apprenticeships. In all, the results suggest that WorkKeys provides useful signals of the 
foundational workplace skills required for success in a work-based learning program. Moreover, 
average WorkKeys scores increased significantly from junior to senior year, which is consistent 
with the notion that students can improve their foundational workplace skills in a work-based 
learning program. 


GPS Education Partners 


The GPSEd program leverages a competency-based curriculum focused on both application and 
foundational knowledge for successful matriculation to a community college, technical college, or directly 
into the workplace. This includes career interest exploration and technical skills development, which offer 
students a full range of career pathways in manufacturing. The program leverages the ACT WorkKeys 
NCRC, the MSSC CPT certificate, and opportunities for dual enrollment. Program elements incorporate 
opportunities to develop social-emotional competencies related to employer-valued skills and attitudes 
needed for success in manufacturing careers. Because students learn apprentice skills, they should be 
better prepared for work or further training after high school. Between 2015 and 2018, the program had 
273 graduates, with 80% of these students pursuing technical careers through immediate employment or 
post-secondary education programs. 
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Students apply to the GPSEd program at the end of their sophomore or junior years. Students who attend 
GPSEd for one year are on track to graduate on time and have an identified career interest upon 
entrance. Students attending GPSEd for two years are often recommended to the program by high school 
counselors, in part, because they are disengaged, credit deficient, and at-risk of not graduating. GPSEd 
students also tend to prefer hands-on learning, have an affinity for technical skills, and find a non- 
traditional school setting appealing. 


Sample and Descriptive Statistics 


The sample analyzed in this study consisted of 94 GPSEd students who graduated in spring 2018 and 97 
who graduated in spring 2019. The sample was predominantly male (90.1%), White (78.5%), and spent 
two years studying at GPSEd (68.6%). In terms of other race/ethnicity groups, the sample was 9.9% 
Black, 4.7% Hispanic, and 1.6% Asian or Pacific Islander. Twenty-two percent of the sample had an 
individualized education plan to help address special learning or attention needs. 


Table 1 provides descriptive statistics for several achievement variables including ACT WorkKeys—an 
assessment of foundational workplace skills. The Wisconsin Department of Public Instruction administers 
WorkKeys to all public high school students in the spring of junior year. GPSEd re-administers WorkKeys 
to their senior students as they approach graduation. The results in this report reflect maximum WorkKeys 
performance, regardless of when the test was administered, because maximum performance is what 
prospective employers or training programs would consider when evaluating applicants. WorkKeys scale 
scores, which range from 65-90, are transformed to level scores with a maximum of 7. The average 
Applied Math score of 79.3 falls in the level 4 range (76—79), the average Workplace Documents score of 
78.9 falls in the level 4 range (77-80), and average Graphic Literacy score of 78.5 falls in the level 5 
range (78-81). 


Table 1. Descriptive Statistics 


25th 75th 
WéeTarele){s) N Mean SD Percentile Median Percentile 
WorkKeys Applied Math 191 79.3 4.0 76.0 80.0 82.0 
WorkKeys Workplace Documents 191 78.9 4.0 77.0 79.0 81.0 
WorkKeys Graphic Literacy 191 78.5 3.5 76.0 78.0 81.0 
CPT Safety 186 85.6 5:3 82.0 86.0 90.0 
CPT Quality Practices & Measurement 175 83.3 5.4 79.0 83.0 87.8 
CPT Manufacturing Processes & Production 124 81.6 5.5 77.0 81.0 86.0 
CPT Maintenance Awareness 110 80.0 5.6 75.0 79.4 84.2 
Lexile Level 189 1167 233 1005 1170 1340 
Class Absences 165 Vere 8.4 2:0 5.0 10.0 
Work Absences 142 48 4.6 2.0 3.0 6.0 


Students who achieve certain levels of performance on ACT WorkKeys Applied Math, Workplace 
Documents, and Graphic Literacy are awarded an NCRC. Specifically, examinees must attain level 3 or 
higher on the three WorkKeys Assessments to earn Bronze, 4 or higher to earn Silver, 5 or higher to earn 
Gold, and 6 or higher to earn Platinum. Figure 1 shows the distribution of level scores for GPSEd 
students on the three WorkKeys Assessments. In the study sample, NCRC levels were distributed as 
follows: 5.2% non-qualifier, 33.0% Bronze, 31.9% Silver, 20.9% Gold, and 8.9% Platinum. 
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Figure 1. Distributions of WorkKeys Level Scores 
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The average MSSC CPT exam scores in Table 1 all exceed the corresponding passing scores, which 
range from 73—76. Sample sizes were smaller for CPT Manufacturing Processes & Production and CPT 
Maintenance Awareness because most students who attend GPSEd for one year attempt only the CPT 
Safety and CPT Quality Practices & Measurement certifications. However, students move through the 
GPSEd academic curriculum at their own pace, so some one-year students complete all four components 
and earn their MSSC CPT certificate. In addition, GPSEd students had an average Lexile reading ability 
level of 1167. For reference, the typical Lexile range for 11th and 12th graders is 1130—1440.? During 
their senior year, GPSEd students were absent from class an average of 7.7 days and absent from their 
apprenticeships an average of 4.8 days. 


Academic Achievement 


Achievement on WorkKeys should correlate positively with other indicators of achievement in similar 
content areas. For example, GPSEd students who earned higher Workplace Documents level scores 
tended to have higher Lexile reading ability levels. Indeed, the correlation between Workplace Document 
scale scores and Lexile levels was .553 (jp < .001), indicating a moderately strong, positive relationship. In 
the past, GPSEd has observed that Lexile reading scores correlated with NCRC levels, earning the CPT, 
and overall program success. In this study, average Lexile levels were consistently higher for GPSEd 
students with higher Workplace Documents level scores (972 for Level 3, 1120 for Level 4, 1284 for Level 
5, 1369 for Level 6, and 1426 for Level 7). 


GPSEd students with higher Applied Math level scores were more likely to have taken higher-level math 
courses during their senior year. Specifically, the percentage whose highest course was Technical Math 
(level 1A or 2), rather than Math for the Trades (level 1 or 2), increased from 0% to 18.0% to 39.3% to 
48.3% to 66.7% as Applied Math level score increased from 3 to 7. Students whose highest course was 
Technical Math scored an average of 3.6 points higher on Applied Math than students whose highest 
course was Math for the Trades. This statistically significant difference (p < .001) corresponded to 1.0 
standard deviation on the Applied Math score scale, which would be considered a large effect. 
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MSSC Certified Production Technician Attainment 


Numerous organizations across the US administer WorkKeys as an indicator of readiness for MSSC CPT 
success. GPSEd does not use WorkKeys for this purpose. Rather, GPSEd considers the NCRC and CPT 
as complementary certifications. Thus, results described below may not generalize to other uses of 
WorkKeys and MSSC CPT. Nevertheless, GPSEd data provide an opportunity to study associations 
between WorkKeys and CPT exam performance. 


As shown in Table 2, WorkKeys had weak positive correlations with the CPT exams, which indicated that 
students with higher WorkKeys scores tended to earn higher CPT exam scores, but the correlations were 
not always statistically significant. Applied Math correlated significantly with CPT Safety (r= .24, p < .01) 
and with CPT Manufacturing Processes & Production (r= .19, p < .05). Workplace Documents correlated 
significantly with CPT Quality Practices & Measurement (r= .17, p < .05) and CPT Manufacturing 
Processes & Production (r= .25, p < .01). Graphic Literacy correlated significantly only with CPT Safety 
(r= .19, p < .01). None of the WorkKeys Assessments correlated significantly with CPT Maintenance 
Awareness. Note that higher correlations between WorkKeys and CPT exams have been observed in 
other studies.* 


Table 2. Correlations Between WorkKeys and CPT Exams 


No} e)i(=\e| MAY Co) ,¢0) f= (er> an 1-10) 0163 


Math Documents Literacy 
CPT Safety .24** 12 .19** 
CPT Quality Practices & Measurement 12 .17* 14 
CPT Manufacturing Processes & Production .19* .25** 17 
CPT Maintenance Awareness 13 .06 13 


*p<.05, p< .01 


High, positive correlations between WorkKeys and CPT exams were not necessarily expected, mainly 
because the assessments measure different constructs. Yet, some degree of association was expected 
since CPT curriculum analyses identified numerous objectives requiring skills measured by WorkKeys.* 
For example, Workplace Documents skills are required for CPT Manufacturing Processes & Production 
(e.g., when reviewing standard operating procedures or identifying product specifications), and Graphic 
Literacy skills are required for CPT Safety (e.g., when interpreting diagrams or locating information on a 
material safety data sheet). 


Considering the small magnitude of the correlations in Table 2, strong associations between NCRC level 
and average CPT exam score were not expected. Indeed, the average CPT exam score tended to 
increase as NCRC level increased from Bronze to Platinum, but it did not increase consistently from one 
level to the next (Figure 2). Note that results for non-qualifiers are not shown in Figures 2 and 3 due to the 
small sample size (fewer than 10 students). 


Pass Rate Percent 
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Figure 2. Average CPT Exam Score by NCRC Level 
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Further analyses focused on the relationship between NCRC levels and CPT pass rates for two groups of 
students: those who attended GPSEd for two years (131) and those who attended for one year but 
attempted all four CPT exams (10). GPSEd puts great emphasis on CPT Safety and CPT Quality 
Practices & Measurement as foundations to support success in the work-based learning activities that 
employers provide the students. Because the pass rates for those exams were both very high (97.9% for 
CPT Safety and 95.0% for CPT Quality Practices & Measurement), there could not have been strong 
relationships between NCRC level and pass rate. Nevertheless, Figure 3 reveals that Bronze NCRC 
earners were least likely to pass those exams. 


The overall pass rates for CPT Manufacturing Processes & Production and CPT Maintenance Awareness 
were 85.1% and 78.0%, respectively. In both cases, those earning higher NCRCs tended to have higher 
CPT pass rates, though the relatively small group of 13 Platinum NCRC earners included a few students 
who did not pass, which reduced their pass rates below those of the Gold and Silver NCRC earners 
(Figure 3). 


Figure 3. CPT Exam Pass Rate by NCRC Level 
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Of the students in this analysis, 76.6% earned the MSSC CPT certification, and students earning higher 
NCRC levels were generally more likely to earn the certificate. Though GPSEd does not use the NCRC 
as a prerequisite for the MSSC CPT, results from this analysis are consistent with the notion that 
WorkKeys measures foundational skills that are important for success in an industry-specific certification 
program. 


WorkKeys Growth 


Recall that all public high school students in Wisconsin take Applied Math, Workplace Documents, and 
Graphic Literacy in the spring of junior year, which allowed for estimating growth in WorkKeys 
performance between spring of junior year and the time at which GPSEd students took the tests again as 
seniors. Examining increases in WorkKeys performance was possible only for 2019 graduates because 
they took the same version of WorkKeys in 2018 and 2019 (2018 graduates took an earlier version of 
WorkKeys in spring 2017). Observing improved WorkKeys scores is a positive finding, and it is consistent 
with the notion that engaging in a work-based, applied-learning environment can increase WorkKeys 
scores. However, the design of this study precludes ruling out other causes such as learning outside of 
school or work and changes in motivation to perform well on the tests. 


As shown in Table 3, the average differences between junior year and maximum WorkKeys scores 
ranged from 2.1 points for Workplace Documents to 3.0 points for Graphic Literacy. In all cases, the 
differences were statistically significant. Those differences corresponded to effect sizes ranging from 0.48 
to 0.69 standard deviations, which would be considered medium effects. The increases in scale scores 
translated to increases in level scores. Specifically, 30.8% increased by one level and 19.1% increased 
by two or more levels on Applied Math, 36.2% increased by one level and 8.5% increased by two or more 
levels on Workplace Documents, and 22.6% increased by one level and 33.3% increased by two or more 
levels on Graphic Literacy. As for NCRC levels, 45.2% increased by one level and 14.0% increased by 
two or more levels. Figure 4 illustrates differences in the NCRC level distributions from junior year to 
students’ maximum performance. Changes in the distribution are most apparent from the decreasing 
percentage of non-qualifiers and the increasing percentages of Gold and Platinum NCRCs. 


Table 3. Mean Difference in WorkKeys Performance between Junior and Senior Year 


Mean Mean Mean Effect 
Assessment iN (Junior) (Maximum) _ Difference Size 
Applied Math 94 76.5 79.4 2.9*** 0.69 
Workplace Documents 94 76.9 79.0 2.1*** 0.48 
Graphic Literacy 93 75.8 78.8 3.0*** 0.67 


#1 ny < 001 
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Figure 4. NCRC Level Distributions over Time 
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Additional Analyses 


Results from several additional analyses are summarized here. 


e WorkKeys performance had a small positive association with apprenticeship pay rate. However, 
apprenticeship pay rate was directly tied to passing CPT exams, so this relationship was 
expected considering the positive association between WorkKeys and CPT exam scores. 


e Other analyses indicated that the typical student who did not qualify for an NCRC at any time had 
10 more class absences during senior year compared to the typical student who earned an 
NCRC (median of 15.5 vs. 5 days). Moreover, there was a weak negative relationship between 
NCRC level and apprenticeship absences. That is, students with higher NCRC levels had fewer 
average apprenticeship absences (5.4 for non-qualifiers, 5.1 for Bronze, 4.7 for Silver, 4.5 for 
Gold, and 4.5 for Platinum). 


e A final analysis compared WorkKeys scores for students with different plans after graduation 
(e.g., manufacturing, military, etc.). The only notable trend was that the 26 students who entered 
full-time postsecondary studies had slightly higher average WorkKeys scores (by 1—2 points). 


Conclusions 


Through collaboration with GPS Education Partners, ACT Research analyzed GPSEd student data to 
examine the relationship between ACT WorkKeys and work-based learning outcomes. Overall, GPSEd 
students exhibited positive program outcomes. Indeed, students exhibited significant improvement on 
WorkKeys from junior to senior year, which is consistent with the notion that students improve their 
foundational workplace skills while at GPSEd. Still, some students might benefit from further skills 
development to support better outcomes by engaging with the WorkKeys Curriculum. Students who took 
higher level math courses at GPSEd tended to perform better on WorkKeys Applied Math, and Workplace 
Document scores had a moderately strong association with Lexile reading ability levels. Students who 
performed better on WorkKeys tended to score higher on their MSSC CPT exams, and they were more 
likely to earn the MSSC CPT certification. In all, GPSEd students who performed well on WorkKeys were 
more likely to have positive academic and certification-related outcomes. Thus, results from the study 
suggest that WorkKeys measures foundational workplace skills required for success in programs like 
GPS Education Partners, which offer work-based learning through apprenticeships. 
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